I-Frame Delay

I-Frame Delay (IFD) is a scheduling technique for adaptive streaming of MPEG video. The idea behind it is that streaming scheduler drops video frames when the transmission buffer is full because of insufficient bandwidth, to reduce the transmitted bit-rate. The characteristics of the algorithm.:

1. number of frames currently in the buffer (not the number of bytes) is indicating buffer fullness,
2. less important frames (B-frame) from the buffer are dropped before the more important frames (I-frame and P-frame),
3. the transmission of I-frames is delayed when conditions are bad, even if they are out-of-date w.r.t. the display time (they can still be used to decode subsequent interpredicted frames).

I-Frame Delay algorithm

The IFD mechanism is divided on two parts:

1. as the stream is parsed and packetized into network packets, it is also analyzed and the packets are tagged with a priority number reflecting the frame type (I-frame, P-frame or B-frame). Non-video packets are given a highest priority number, which causes that audio will never be dropped.
2. during transmission, packets are dropped by the IFD scheduler when the bandwidth is insufficient.

The size of the IFD buffer should be big enough to hold a number of frames but minimum required is two frames, one to hold the frame currently being sent (indicated below as ScheduledFrame), and one currently waiting to be sent (indicated as WaitingFrame). Increasing the IFD buffer size could potentially permit a more elaborate prioritization, however it can cause increased latency and memory usage. The figure below depicts an example of the buffer filling. The numbers represent the priority a packet.

Here the video frames priority numbers are 10 and higher. The packets with priority number 12 belong to the frame scheduled for sending, and the packets with number 11 belong to the waiting frame. On the figure a packet which belongs to the next frame is about to enter the buffer.

As can be seen, it is possible to interleave video packets with non-video packets (audio or system with priority numbers 2 and 0 respectively). When a packet belonging to next frame is about to be written to the IFD buffer and the buffer is full, the IFD scheduler will drop a frame based on the priority assigned earlier. When the network bandwidth is so low that also P-frames need to be dropped, then the GOP (Group of Pictures) is set to be "disturbed" and the rest of the GOP (which depends on the P-frame) is also dropped.

If only B-frames are dropped there should be no distortions in the frame image because there are no subsequent frames depending on them. The dropping of frames by IFD causes the effect of the video playback being temporarily frozen, the duration of which depends on the number of frames dropped after which the playback resumes from the next frame which got through. For an IFD implementation with a buffer of the size of two frames the algorithm is shown in figure below.

procedure Enqueue(NextFrame)
  if DisturbedGOP == True then
    if NextFrame is type I then             # New GOP is encountered
      DisturbedGOP = False                  # Reset disturbed GOP flag
    end
  end
  if DisturbedGOP == True then
    Drop NextFrame                          # Discard rest of disturbed GOP
    return
  end
  if WaitingFrame is empty then
    WaitingFrame = NextFrame
  else
    if NextFrame is type I then
      WaitingFrame = NextFrame
    else
      if NextFrame is type B then
        Drop NextFrame
      else
        if WaitingFrame is type I or P then
          Drop NextFrame
          if NextFrame is type P then       # Discarded frame is P-frame
            DisturbedGOP = True             # Set disturbed GOP flag
          end
        else
          WaitingFrame = NextFrame
        end
      end
    end
  end
end